
THE INTERNATIONAL 
REVIEW OF RESEARCH IN 
OPEN AND DISTANCE LEARNING 


Using Supplementary Video in Multimedia 
Instruction as a Teaching Tool to Increase 
Efficiency of Learning and Quality of 
Experience 



Milos Ljubojevic 1 , Vojkan Vaskovic 2 , Srecko Stankovic 3 , and Jelena Vaskovic 2 

Academic and Research Network of Republic of Srpska, BiH, University of Belgrade, Serbia, 3 City Hall of 
Banja Luka, BiH 


Abstract 


The main objective of this research is to investigate efficiency of use of supplementary 
video content in multimedia teaching. Integrating video clips in multimedia lecture 
presentations may increase students’ perception of important information and 
motivation for learning. Because of that, students can better understand and remember 
key points of a lecture. Those improvements represent some important learning 
outcomes. This research showed that segmentation of teaching materials with 
supplementary video clips may improve lecture organization and presentation in order 
to achieve effective teaching and learning. The context of the video content and the 
position of supplementary video clips in teaching material are important influences on 
factors for motivation and efficiency of learning. This research presents the effects of the 
use of supplementary videos with different context of content (entertainment and 
educational) as well as the effects of their position within the teaching material. The 
experimental results showed that the most efficient method of use of supplementary 
video is integration with educational video content in the middle of a lecture. This 
position of video insertion provides the best results. The context of video content 
influences efficiency of learning also. Entertainment video was not as efficient as 
educational, but it can be used to engage and motivate students for learning. The given 
results have been confirmed with a subjective assessment of students’ quality of 
experience with different methods of embedding video clips. 
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Introduction 


Web-based instruction in the educational and training domains plays an important role, 
and its effects on learning outcomes, performance, and student satisfaction are 
important research topics (Olson & Wisher, 2002). Integration of video clips in teaching 
materials has recently attracted more attention in academic research. 

The outcomes of using supplementary videos are increasing student activity and 
efficiency of the teaching process. An important issue is establishing a methodology of 
embedding video clips in multimedia teaching material in order to improve the learning 
process (Kay, 2012). Distance learning and college courses where the students are 
working individually on their projects after class are especially suitable for testing of 
new methods aimed at improvement of students’ work. 

The main motivation for testing the effects of video clip integration into multimedia 
presentation is that visual stimulation with a media application familiar to the student 
population can increase their engagement. It is well known that Internet video traffic is 
constantly increasing its share of consumer Internet traffic. Social media applications 
such as Facebook, YouTube, blogs, and wikis can be used as supplemental materials in 
the teaching process (Burke, Snyder, & Rager, 2009). 

The selection of appropriate video clips and methodology for their display within the 
teaching materials represents an important issue for curriculum design, leading to 
positive learning outcomes (McConville & Lane, 2006). The cognitive theory of 
multimedia learning shows that it is necessary to select relevant information and 
organize it into a verbal and pictorial model (Mayer, 2001). Using appropriate teaching 
media and methods to organize and present only relevant information may also increase 
the efficiency of the self-learning process (Ruiji, 2012). Evaluating the effectiveness of 
the use of teaching materials in video format in distance learning environments and the 
measuring of “viewer engagement” is important for improvement of the learning 
process (Stiubiener et al., 2012). 

An important aspect of the learning process is students’ satisfaction, especially in the 
case of online learning (Roach & Lemasters, 2006). Considering this theory, students’ 
satisfaction, as an important outcome of a multimedia learning process, should be 
analyzed more closely. The quality of student satisfaction and experience depends on 
the method of usage of video clips in designing linear educational video materials. 
Therefore, the quality of the learning process may be described also with quality of 
experience (QoE). 
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In this paper, the efficiency of learning and quality of experience for different types of 
video clips and methods for integration in multimedia lecture presentations are 
analyzed. 

Video as a Factor of Influence in Motivation, Satisfaction, and 
Successful Learning 

Modern teaching processes are based on the use of multimedia teaching materials and 
the Internet. The cognitive theory of multimedia learning represents a foundation for 
the implementation of multimedia educational content presentation. This theory 
explains the significance of the modality principle in the learning process, as shown in 
Figure 1. 

In order to efficiently process multimedia information it is necessary to select relevant 
information and organize it into a verbal and pictorial model. Processing the 
information is performed in two channels after receiving the auditory and visual 
information that appears in the working memory as verbal and pictorial information 
models. A rational use of resources is very important, so design of multimedia 
presentation is a crucial factor for the learning process outcome (Mayer, 2001; Moreno 
& Mayer, 1999). 



Figure 1. The cognitive theory of multimedia learning (Mayer, 2001). 


Cognitive load theory, educational research, and instructional design are closely related, 
thus they should be analyzed together to achieve the best learning results (De Jong, 
2010). Previous research also shows that the efficiency of the learning process is closely 
related to interaction between motivational and cognitive variables (Valle et al., 2003). 

Multimedia technology has been exploited often for improving teaching and learning. 
Videos are a tool for engaging the verbal (linguistic), visual (spatial), and musical 
(rhythmic) intelligence of the student in the learning process, especially in the self¬ 
learning process (Gardner, 2000). Positive effects of features of video clips (multi- 
sensory, dynamic and capable of engaging the viewer's attention) were often used in 
engineering education (Marques , 2012). 
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Different methods of embedding and different contexts of supplementary video content 
can be used in the teaching process. Teachers are using multimedia presentation 
prepared for the course as well as additional, supplementary videos. Supplementary 
videos may have different contexts. Two of them are important for this research: The 
first is that they are closely related to the presented lecture topic and the second context 
is entertainment used for capturing the attention and interest of the students. Methods 
for integration and use of different contexts of supplementary videos in multimedia 
education materials are important tasks for research aiming to improve the learning 
process. 

In this research, the purpose of supplementary video material use is to increase 
student’s attention on the topic of the lecture. In order to increase learning quality, two 
important goals should be achieved: reducing cognitive load and increasing student's 
attention. Besides educational and entertainment content, it is possible to use 
commercial content as supplementary video. Although the commercial content may 
contribute to reducing the cognitive load of the student, this type of context potentially 
may take away students’ attention from the topic of the lecture and focus it on the 
commercial content. Due to that, supplementary videos with commercial content have 
been less relevant to this research than educational and entertainment content. In 
future research, the influence of supplementary videos with commercial content on 
students’ attention should be investigated. 

Previous research explored the results of the use of videos as an educational tool (Bravo 
et al., 2011). In this study the authors used educational videos with a duration of 
approximately four minutes. They used a teaching platform based on Moodle and 
YouTube channels for reproduction. The study showed that streaming videos used as 
supporting material for learning had a positive effect upon students’ perception 
regarding the enhancement of their learning motivation. The study emphasized that 
proper definition of content and amount of information delivered through supplemental 
video is an important task to solve. Durations of videos and methodologies of displaying 
them influence the effectiveness of learning and student satisfaction. An important 
result of this research is that streaming supplemental videos improve students’ ability to 
learn in an autonomous way. 

Hsin and Cigas (2013) used short videos to enhance student satisfaction and motivation 
for an online introductory course in computer science/mathematics. They achieved a 
significantly higher percentage of involved students and their average grades increased. 
Another method to engage and to motivate students is showing entertainment videos 
(Steffes & Duverger, 2012). As Steffes and Duverger reported, showing supplementary 
videos within an entertainment context at the beginning of the class can be used to 
increase the positive mood of the students. Both previous studies, Bravo et al. (2011) 
and Steffes and Duverger (2012), emphasize the importance of the proper design of 
supplementary video content and the methodology of displaying video to students. 
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Donkor (2011) showed that use of video-based instructional materials for teaching 
practical skills at a distance also appears promising. 

One of the key questions is the quality of experience (QoE) achieved with the presented 
multimedia materials. This problem is more interesting if the different contexts of the 
video content are merged. In the research presented in Le Callet et al. (2012), the 
authors defined experience as an individual’s stream of perception and interpretation of 
one or multiple events. On the other hand, quality is defined as an outcome of an 
individual’s comparison and judgment process, which includes perception, reflection 
about the perception, and description of the outcome. 

Finally, a working definition of quality of experience is: 

Quality of Experience (QoE) is the degree of delight or 
annoyance of the user of an application or service. It 
results from the fulfillment of his or her expectations 
with respect to the utility and/or enjoyment of the 
application or service in the light of the user’s 
personality and current state. (Le Callet et al., 2012) 

The definition of quality of experience gives a framework for analyzing the efficiency of 
delivery of video content using the Internet. Ljubojevic et al. (2013) analyzed linear 
internet video advertising efficiency using QoE methods. Another important task is the 
influence of the delivery of supplementary video used in the teaching process on 
students’ quality of experience. 

Therefore, students’ quality of experience and learning outcome may depend on the 
context of inserted supplementary videos and the method for embedding video into 
teaching material. 


Method 


Previous research findings showed that segmentation of multimedia teaching materials 
enables easier processing of complex information by reducing high cognitive load. 
Students with a low level of prior knowledge learned more efficiently if segmented 
material was presented instead of continuous material (Spanjers et al., 2011). This 
finding is especially important for novices in the learning process. 

In our study, the participants in the experiment were students who attended the class 
for the first time and had a lower level of prior knowledge. Therefore, the efficiency of 
different methods of segmentation of multimedia teaching material in learning for 
novices was analyzed. Due to the fact that video with different video content types is 
inserted in multimedia learning materials, supplementary videos have a different 
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influence on students’ motivation and QoE. The efficiency of different methods for 
integrating those videos should be investigated in detail. 

The teacher has to decide how he wants to use video clips, where and when to embed 
them into multimedia learning material (Berk, 2009). Previous research described the 
effectiveness of videos embedded in multimedia teaching materials and their influence 
on the improvement of teaching (Pryor & Bitter, 2008; Borko et ah, 2008). This choice 
has a strong influence on the students’ QoE and the efficiency of the learning process. 
The instructional design model is also important in developing teaching skills (Isman et 
ah, 2012). In order to analyze the multimedia learning quality process and level of 
student motivation for learning, especially if supplementary video streaming was used 
within the lecture, the quality of experience assessment should be used. 

Testing Methodology 

Important aspects of this study are principles for designing multimedia teaching content 
and presenting supplementary videos. A method for assessment of the efficiency of 
learning and student’s QoE was also proposed. 

In this study two influence factors in motivation and efficiency of learning were tested: 
the context of inserted supplementary video and position of the embedded videos into 
teaching material. 

Two contexts of videos were used: the entertainment context of the video and the 
educational context similar to a lecture. Methods for embedding the videos were 
analyzed in the sense of the position of the supplementary video regarding the main 
multimedia teaching material. Three possible methods were tested: displaying 
supplementary videos before, in the middle, and after the lecture. 

In order to simulate spatial and temporal separation between the students and the 
professor (main characteristic of distance learning and the self-learning process), 
multimedia materials created for the experiment were presented to the students in a 
multimedia classroom. Two assistants also assisted as technical support during the 
lecture presentation. 

Each of the applied methods for embedding supplementary videos was separately 
tested. After the lecture presentation, using one of the methods, participants had a 60 
minute break and then the testing was performed. Students answered 20 questions 
related to the lesson that was presented. By analyzing the given testing results, the 
efficiency of different methods for inserting supplementary video may be compared. 

A subjective assessment of the user’s quality of experience was also conducted. The 
testing was performed according to the subjective video quality assessment methods for 
multimedia applications described by Recommendation ITU-T P.910 (ITU-T, 1999). 
During the testing, a single test condition (one multimedia education content with 
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supplementary video) was presented to students once only. Then they gave a quality 
rating on an Absolute Category Rating (ACR) scale. By use of the previously described 
methodology, the subjective assessment of the quality of experience (QoE) was 
conducted immediately after the presentation of the teaching materials. 

In order to assess the subjective perception of the method for the presentation of lecture 
content, participants answered carefully chosen questions, listed in Table 1. 

Table 1 

Assessment Questions 


No. 

Question 

1 

Were the directions for participating in experiment sufficient? 

2 

Did you find the quality of presenting a lecture sufficient? 

3 

Did you think video and audio quality of supplementary video 
stream was good? 

4 

Were you able to maintain the attention level during the session? 

5 

Do you think that you correctly answered the questions? 

6 

Was it easy to answer the questions? 


According to Recommendation ITU-T P.910 and assessment and rating of the quality of 
experience, a five level mean opinion score (MOS) scale was used as presented in Table 
2. 

Table 2 

Scale for Quality of Experience Rating and Gradation 


Mean opinion score (MOS) 

Explanation 

5 

Excellent 

4 

Good 

3 

Fair 

2 

Poor 

1 

Bad 


This research has several limitations, which are related to deeper analysis that needs a 
longer period of testing. The performed experiment should be applied for a whole 
semester with the participation of a number of different classes of students. Several 
different curriculums should be used in research for a deeper analysis and an efficient 
comparison of results. Those limitations should be overcome in future research. 
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Participants and Testing Environment 

The research was performed at a university located in Bosnia and Herzegovina. The 
multimedia classroom is well-equipped for advanced multimedia presentations and 
high speed internet access. 

The participants were 46 undergraduate students who attended the same academic 
course. During the preparation of the experiment students were informed about the 
testing procedure. In order to motivate students for serious participation they were 
informed that they would receive extra credits for their course grade based on their 
individual performance. Participants filled in questionnaires regarding personal data, 
prior experience related to multimedia learning methods, and prior knowledge about 
the topics covered in the multimedia testing materials. 

Analysis of collected data showed that participants had a similar level of prior 
knowledge about the multimedia learning and topics that were presented in the 
multimedia teaching materials. Therefore, it may be assumed that there was 
homogeneity in learners’ prior knowledge and skills related to the teaching process. 
Demographic data showed that out of 46 participants, 28 were male and 18 were female 
with the average age under 22 (93%). 

Design of the Testing of Multimedia Learning Material 

The multimedia teaching material was designed for the purpose of this research using 
topics defined in the course curriculum. The teaching material consisted of two parts: 
the main multimedia educational material and the supplementary videos. The main 
multimedia education material was created using video lectures recorded by using a 
professional video camera and PowerPoint presentations. Video lectures and 
PowerPoint presentations were merged so the text of the PowerPoint presentation, 
audio information and effect of occasional appearance of the instructor on the screen 
were used to achieve better knowledge transfer. 

Supplementary videos that were used were downloaded from YouTube keeping in mind 
users’ interests and the number of views. Video clips were inserted and presented 
before, in the middle, and after the main multimedia educational material. The method 
of integrating two videos by displaying only one of them on the screen, if the user has no 
control over the presented video, may be described as a linear method of multimedia 
presentation. The supplementary videos have two contexts. The first context is 
entertainment and the second one is closely related to the lecture. 

For designing one dataset of teaching material one main multimedia educational 
material content and two videos with previously described contexts (entertainment and 
educational) were used. From the aspect of usage of supplementary video, three types of 
multimedia test material were created: 
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• Supplementary videos were not used at all and the lecture was presented 
continuously; 

• Supplementary video with the entertainment context was used; 

• Supplementary video with the educational context closely related to the lecture 
topic was used. 


In order to test the influence of the position of supplementary video within the 
multimedia presentation, the video was displayed at the beginning, in the middle, and at 
the end of the main multimedia educational material, as shown in Figure 2. Therefore, 
for each type of multimedia materials three different subtypes were created. 



\ Main multimedia educational material J 


Figure 2. Principle of inserting the supplementary video in lecture presentation. 


According to the proposed methodology the following types of multimedia teaching 
materials were created: 

• main multimedia without supplementary video and application of continual 
presentation of lecture (M_nv); 

• main multimedia with educational supplementary video inserted at the 
beginning (M_ed 1), middle (M_ed 2), and at the end of the presentation 
(M_ed 3); 

• main multimedia with entertainment supplementary video inserted at the 
beginning (M_ent 1), middle (M_ent 2), and at the end of the presentation 
(M_ent 3). 

The influence of the segmentation method by inserting supplementary video within the 
presentation of the lecture content was analyzed comparing the previously described 
methods of lecture presentation. 
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Results 


For every type of multimedia education material which was created for this research, the 
level of acquired knowledge and student's quality of experience were tested. In order to 
describe key research variables, two research instruments were used. Test questions and 
results illustrated with average number of correct answers, median value, and standard 
deviation were used to measure the level of the acquired knowledge and to describe the 
testing procedure. The assessment of students’ quality of experience was performed 
using a MOS scale with the five levels. 

If the presentation of the lecture is continuous, without inserting supplementary videos, 
which is considered as a method that provides acquiring a reference level of knowledge, 
two remaining methods may describe possible improvements achieved by inserting 
different videos. 


Tested methods of embedding the supplementary videos were: 

• main multimedia without supplementary video using continual lecture 
presentation (M_nv); 

• main multimedia with educational supplementary video inserted at the 
beginning (M_ed 1); 

• main multimedia with educational supplementary video inserted in the middle 
(M_ed 2); 

• main multimedia with educational supplementary video inserted at the end of 
the presentation (M_ed 3); 

• main multimedia with entertainment supplementary video inserted at the 
beginning (M_ent 1); 

• main multimedia with entertainment supplementary video inserted in the 
middle (M_ent 2); 

• main multimedia with entertainment supplementary video inserted at the end 
of the presentation (M_ent 3). 

The results of testing the acquired knowledge and student's quality of experience are 
analyzed and presented in Table 3 and Table 4, respectively. In Table 3, the average 
number of correct answers for every tested method of embedding and displaying 
supplementary videos is presented. 
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Table 3 

Results of Testing of Acquired Knowledge 




Methods of embedding and displaying supplementary video 

Statistical 

measure 


M nv 

M ed 1 

M ed 2 

M ed 3 

M ent 1 

M ent 2 

M ent 3 

AV 

12.22 

13.78 

15-50 

12.85 

13-13 

13.76 

12-33 

Med 

12.00 

14.00 

15.00 

12.00 

13.00 

14.00 

12.00 

Std 

3 -H 

2.51 

2.29 

2.52 

2.00 

2.06 

2-44 


AV - Average number of correct answers, Med - Median value, Std - Standard deviation 


The least number of correct answers was in the case of continual presentation of lecture 
content. It illustrates that segmentation by inserting video clips may increase memory 
and better understanding in the learning process. Therefore, the results showed that 
methods based on the insertion of videos in multimedia teaching materials gave a 
higher number of correct answers regardless of the position or the context of 
supplementary videos. 

The presented results indicate that a more efficient method for increasing the students’ 
participation and deeper learning is to display the educational context of supplementary 
video clips instead of inserting entertainment videos. 

The experimental results also showed that the biggest influence on efficiency of learning 
is the method that is based on inserting videos in the middle of the lecture presentation. 
The results were the same for both contexts of the supplementary videos. In the case of 
inserting the videos in the middle, the data were closest to the mean and the standard 
deviation was smallest, so the answers were fairly uniform. It confirmed the superiority 
of inserting video clips in the middle of the lecture. 

The results of knowledge testing (average number of correct answers) for different 
methods of inserting supplementary video clips are illustrated in Figure 3. 
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Average number of correct answers 

18 00 -I- 



Figure 3. Illustration of average number of correct answers. 


Experimental results of the subjective assessment of quality of experience for every 
tested method of embedding supplementary video, presented in Table 4, showed that 
participants were satisfied with the preparation of the experiment process, technical 
aspects of the presented material, and quality of the lecture presentation. 

Table 4 

The Results for Assessment of Quality of Experience 



MOS values 


M nv 

M ed 1 

M ed 2 

M ed 3 

M ent 1 

M ent 2 

M ent 3 

Question 1 

5 

5 

5 

5 

5 

5 

5 

Question 2 

5 

5 

5 

5 

5 

5 

5 

Question 3 

5 

5 

5 

5 

5 

5 

5 

Question 4 

4-t 

4-7 

4-8 

4-5 

4.2 

4-4 

4.2 

Question 5 

3-4 

4 

4.2 

3-9 

3-6 

3-7 

3-5 

Question 6 

3-9 

4-3 

4-3 

4-1 

4-t 

4.2 

4 


The attention of participants was maintained at the highest level during the session with 
the use of educational supplementary videos, especially if they were inserted in the 
middle of the presentation. The usage of complementary video with an educational 
context gave the best results regarding students’ self-confidence in the answering of test 
questions. 

An illustration of the MOS results for questions 4, 5 and 6, that closely describe quality 
of experience, is presented in Figure 4. 
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Figure 4. An illustration of MOS results for questions 4, 5 and 6. 


Demographic data analysis confirmed that there were slight differences between male 
and female students in terms of quality of experience with e-learning. Female students 
were a little more satisfied if supplementary video material was used within the lecture 
presentation. 


Conclusion 


This research aims to present the positive effects of the use of videos as a supplementary 
teaching tool. Students’ motivation and efficiency of learning may be increased if 
supplementary videos are used. 

The analysis of methods for inserting video clips regarding different positions and the 
context of supplementary content shows that the methodology of inserting the 
supplementary videos is very important for improvement of teaching. 

The positive effects of segmentation as an instructional method of teaching was 
confirmed with experimental results. Students acquired a higher level of knowledge if 
the lecture was presented in segments separated with videos, regardless of the context of 
the video content. 

Research also demonstrated the importance of adequate design and presentation of 
multimedia teaching materials. Different contexts of use of supplementary video 
content in teaching showed that better efficiency is achieved if the educational content 
congruent with the lecture has been displayed. Entertainment video was not as efficient 
as educational, but it can be used to engage and motivate students’ learning. 
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The position of inserted video has an important role in increasing efficiency of learning. 
The entertainment video at the beginning of the lecture motivated students towards 
learning, so testing results were better for that method than if a lecture without videos 
was used. A better effect was achieved when educational video content was used. This 
research showed that the most efficient method of use of supplementary video is 
integration of educational video content in the middle of the lecture. 

Positive feedback from students about enhancement of their learning motivation and 
results of performed testing confirmed positive effects of the use of videos in teaching. 
Subjective assessment of quality of experience achieved by the use of supplementary 
video in teaching confirmed that the educational video inserted in the middle of the 
lecture may increase efficiency of learning. The results indicate that female students are 
more interested than male students to use learning material enriched by supplementary 
videos. 

In further research we are planning to investigate different aspects of integrating video 
content into a presentation (duration of video, usage of video streaming, interactive 
video content, etc.). 
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